arXiv:1508.05217v2 [math.OC] 22 Sep 2015 


Co-design of controller and routing 
redundancy over a wireless network^ 

G.D. Di Girolamo, A. D’Innocenzo and M.D. Di Benedetto 

Department of Information Engineering, Computer Seienee and 
Mathematies, Center of Exeellenee DEWS, Univ. of L’Aquila, Italy 


Abstract: In this paper we investigate the exploitation of redundancy when routing actuation 
data to a discrete-time LTI system connected to the controller via a wireless network affected 
by packet drops. We assume that actuation packets can be delivered from the controller to the 
actuator via multiple paths, each associated with a delay and a packet loss probability. We show 
that the joint design of controller gain and routing redundancy exploitation can tremendously 
improve the control performance. To achieve this goal we set up and solve a LQR problem 
for a class of systems that extends discrete-time Markov Jump Linear Systems, in that both 
continuous and discrete control signals can be actuated. 
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1. INTRODUCTION 



Wireless control networks (WCN) are distributed con¬ 
trol systems where the communication between sensors, 
actuators, and computational units is supported by a 
wireless communication network. The use of WCN in in¬ 
dustrial automation results in flexible architectures and 
generally reduces installation, debugging, diagnostic and 
maintenance costs with respect to wired networks (see 
e.g. I.F. Akyildiz and I.H. Kasimoglu (2004), Han et al. 
(2010) and references therein). However modeling, analysis 
and design of (wireless) networked control systems are 
challenging open research problems since they require to 
take into account the joint dynamics of physical systems, 
communication protocols and network infrastructures. Re¬ 
cently, a huge effort has been made in scientific research 
on NCSs, see e.g. Astrom and Wittenmark (1997), W. 
Zhang et al. (2001), G.C. Walsh et al. ( 2002 ), K.- 
E. Arzen et al. (2006), Tabbara et al. (2007), J.R 
Hespanha et al. (2007), Gupta et al. (2009), M.G.F. 
Donkers et al. (2011), R. Alur et al. (2011), Pajic et al. 
(2011), DTnnocenzo et al. (2013) and references therein 
for a general overview. 


^ The research leading to these results has received funding from 
the Italian Government under Cipe resolution n.l35 (Dec. 21, 2012), 
project INnovating City Planning through Information and Com¬ 
munication Technologies (INCIPICT). 


To make a WGN robust to packets losses redundancy in 
data routing can be used. One approach to exploit this 
redundancy is relaying data via multiple paths and then 
appropriately combining them, which is reminiscent of 
network coding. In Smarra et al. (2015) we considered a 
state-feedback control loop as in Figure 1 where multiple 
copies ui{k) = U 2 {k) = ... = Ur{k) = Kxp{k) of 
the same actuation data are sent from the controller 
to the plant via r different routing paths {piYi=i each 
characterised by a delay di and a packet losses probability 
Pi. We assumed that the time-invariant controller gain K 
is designed via classical methods to assign the eigenvalues 
of the closed-loop system in the nominal case, i.e. when 
the effect of packet losses is not considered. We also 
assumed that the actuator computes a linear combination 
Yll=ili'^i{^ — di) of the data incoming from different 
routing paths, and we provided a suboptimal algorithm 
to compute the optimal weights 7 ^ that maximize a metric 
induced by the notion of Mean Square Stability. In this 
paper we continue the research line started in Smarra 
et al. (2015) and provide novel results that strongly 
improve the controller performance. The first difference 
is motivated by the following consideration: when routing 
redundancy is exploited in communication systems the 
objective is to relay some information, and thus we send 
to the network the same packet ui{k) = ... = Ur{k) and 
try to extract from the corrupted received packets the 
original information; in our case the objective is to increase 
the control performance, as a consequence the actuation 
packets ui{k),, Ur{k) must not be necessarily equal. In 
this paper we perform eontroller and routing redundaney 
eo-design by designing the time-varying matrix 3 

K{k) = [Ki{k) G ]Kr{k) G Note 

that the problem formulation in Smarra et al. (2015) 
is a special case of the above definition where K(k) = 
[71 A*;-- - ; 7 rA*] and A* is designed for the nominal 
case. As a further improvement, while in Smarra et al. 
(2015) we optimise a metric based on the notion of Mean 
Square Stability (i.e. only taking into account the steady 





















state behavior) we consider here a more complex control 
specification that also takes into account the transient 
behavior by setting up a finite-horizon LQR problem. In 
Mesquita et ah (2012) the authors also consider redundant 
data transmission over a set of paths characterised by 

i.i.d. Bernoulli probabilities of packet losses, but for a 
more restricted scenario with respect to ours because they 
assume that all paths are associated with the same delay, 
the packet loss events are measurable, the controller is 
designed for the nominal case (i.e. without considering the 
effect of packet losses) and redundant data combination 
is not modeled/designed. Their focus is on deciding how 
many redundant copies of a packet should be transmitted 
at each sampling time and what benefits can be drawn 
from this: besides the fact that our model is more general, 
we also address the more general problem of co-designing 
controller and routing redundancy. 

In Section 2 we define a network modeling framework 
that allows co-design of controller and routing redundancy 
while taking into account the effect of packet losses and in 
Section 3 we provide our LQR problem formulation. 

In Section 4 we assume that the set of paths used at each 
time instant to send actuation data to the actuator has 
be designed a priori: we will call this approach static rout¬ 
ing redundancy. We setup the problem of co-desiging the 
controller gain and the routing redundancy parameters as 
a LQR problem for a Discrete-Time Markov-Jump Linear 
System (dtMJLS) where the discrete mode (which correp- 
sond to the occurrence of packet losses) is unmeasurable 
and evolves according to a sequence of i.i.d. random vari¬ 
ables. The latter assumption, widely adopted for several 
communication systems, makes particularly sense in our 
framework since exploitation of redundant data is well 
known to be very effective especially in the case when 
the reduntant paths used for data relay are uncorrelated 
from the point of view of the communication channels’ 
characteristics. The proof for solving the LQR problem for 
such a system, being the discrete state unmeasurable, is an 
extension of the solution for dtMJLS in Costa et al. (2005) 
and can be derived without much difficulty thanks to the 

i.i.d. assumption. Note that a similar problem has been 
addressed in Matei et al. (2008), with the assumption that 
the discrete state is measurable with a one step delay, and 
solved without proof: we believe that the proof in this case 
is very close to ours, but since we were unable to find it in 
the scientific literature we provide one in this paper for the 
sake of completeness. We apply our optimal solution to a 
simple example where actuation data can be sent to the ac¬ 
tuator via two paths: the first characterised by short delay 
(i.e. fast reaction to perturbations) and high probability of 
packet losses (i.e. low reliablity); the second characterised 
by long delay (i.e. slow reaction to perturbations) and 0 
probability of packet losses (i.e. perfect reliablity). Note 
that such situation can often occur in realistic cases: one 
example is a multi-hop wireless network where we can 
reach the destination via a single long hop (short delay, 
high packet loss probability) or via a path of very short 
multiple hops (high delay, low packet loss probability); 
another example is a service provider network where we 
can reach the destination via the shortest yet congested 
path of routers (short delay, high packet loss probability) 
or via a longer uncongested path of routers (high delay, low 


packet loss probability). In the above situations, using only 
the first path is clearly not a good idea since the closed loop 
system may easily become unstable. Using only the second 
path is the optimal solution to maximise bandwidht, i.e. 
optimal from the point of view of communication theory: 
however, due to the high delay, the control system is not 
reactive to perturbations. The main idea that motivates 
this paper is based on the intuition that we could use both 
paths simultaneously, exploiting the fast reaction of the 
first path and the high reliability of the second path in 
an optimal way taking into account the plant dynamics. 
Our Monte Carlo (MC) simultations show that routing 
actuation data on both paths simultaneously and applying 
our optimal solution, we can tremendously improve the 
performance of both single-path solutions from the point 
of view of control performance. 

In Section 5 we consider a much more complicated prob¬ 
lem: we assume that the set of paths used at each time 
instant to send data to the actuator can be controlled, 
i.e. the choice of redundant routing paths is also a control 
variable: we will call this approach dynamic routing redun¬ 
dancy. We setup the problem of co-desiging the controller 
gain, routing redundancy parameters and paths as a LQR 
problem for a class of systems that includes dtMJLS as a 
special case and provide, as the main theoretical contri¬ 
bution of this paper, a recursive solution that is optimal 
for a certain set of initial conditions, which we define in 
closed form. More precisely, our model is a dtMJLS where 
we can also apply a discrete control that, choosing some 
of the system matrices, models the choice of the routing 
at each time step. A similar model has been considered 
in Vargas et al. (2010), where only a sub-optimal solution 
is provided based on a conservative approximation (see 
the proof of this paper for more details). Similar problems 
have also been considered, but for determinstic models, in 
Zhang et al. (2009) and Borrelli et al. (2005). Finally, in 
Gatsis et al. (2014) a different LQR optmisation problem 
is considered where the discrete and continuous control 
signals are independent and can be designed separately, 
which is not the case in our problem. 

2. NETWORK MODELING 

Gonsider a state-feedback networked control loop as in Eig- 
ure 1 where the communication between the controller and 
the actuator can be performed via a set of r routing paths 
{piYi=i in a wireless multi-hop communication network. 
Each path pi is characterised by a delay di G N+ and 
a packet loss probability pi G [0,1] that represents the 
probability that the packet transmitted on that path will 
not reach the actuator due to communication failure. In 
particular, we define for each path pi the stochastic process 
(Ji{k) G {0,1}, with (Ji{k) = 0 if the packet expected to 
arrive via the routing path pi at time k suffered a packet 
drop and (Ji{k) = 1 if the packet is succesfully received 
at time k. We assume that (Ji{k) is a sequence of i.i.d. 
random variables, each characterised by a Bernoulli dis¬ 
tribution with probability measure ¥[ai{k) = 0] = Pi. We 
also assume here that the events of occurrence of packet 
losses in the different paths are i.i.d.: as a consequence the 
stochastic process a{k) = [cri(/c),..., cr^(/c)]' is a vector of 

i.i.d. random variables, where a{k) can assume 2^ values. 
We also assume that the controller cannot measure the 




signal cr(/c), i.e. it is not possible to measure the occurrence 
of packet losses. The case when the occurrence of packet 
losses is measurable with finite delay thanks to acknowl¬ 
edgement packets will be investigated in future work. We 
assume that, in general, the controller can decide for each 
time instant k the set of paths where data will be sent: 
i.e., the controller can decide to send data at time k on 
all paths, on a subset of paths, on one path, or even not 
to send any data. To this aim we define for each path i 
the discrete control signal ai{k) G {0,1}, with ai{k) = 1 
if the controller decides to send a packet via the routing 
path i at time k, and ai{k) = 0 if no packet is sent via 
path i at time k. We define the discrete control signal 
a{k) = [ai(/c),..., ar(/c)]', where a{k) can be choosen 
among 2^ different values. 

Let the plant be a discrete-time LTI system described by 
the matrices Ap G , Bp G we define the 

dynamics of the networked system as follows: 

(x{k + l) = A^,^k)x{k) + Ba(k)u{k) ,, 

\y{k)=x{k) 


3. PROBLEM FORMULATION 


The network modeling framework in the previous section 
is a special case of the following mathematical framework, 
which is the one we will use in the rest of the paper: 

(x{k + l) = A^,^k)x{k) + Ba(k)u{k) 

\y{k)=xik) ’ 


where x{k) G W^,u{k) G G {l,...,g} = 

T^,a{k) G {l,...,p} = A, and a{k),k > 0 is a sequence 
of i.i.d. random variables such that ¥[a{k) = i] = tt^ for 
any i G S, /c > 0. 

Problem 1. Given System (2), design for any /c G {0,..., N— 
1} an optimal state-feedback control policy {x{k))^u^ {k) = 
K'^{x{k))x{k), with a*(x(/c)) : ^ ^4 and K'^{x{k)) a 

m X n matrix of reals, minimizing the following objective 
function: 


where is the sigma algebra generated by x(0), ...,x{k). 
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4. CO-DESIGN OF CONTROLLER AND STATIC 
ROUTING REDUNDANCY 


In this Section we address Problem 1 assuming that the 
routing has beed designed a priori. 

° Theorem 1. Given System (2) and a routing policy defined 

0 £+i.(r)xmr ^ = clr E A, thc Optimal solution of Problem 

^ ’ I is given by a sequence K''{k) with /c = 0,..., A" — I. 


with 
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Ai(a{k)) = ai{k) ^Bp 0 o] 
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and with v(i) = m ^ dj^ Im the m-dimensional identity 
i=i 

matrix, a column vector of appropriate dimension with 
all zero entries except the i — th entry equal to I, and 
(g) the Kronecker product. System (I) is more general 
than dtMJLSs: in particular, when V/c > 0, a{k) = 

(i.e. the routing is designed a priori) System (I) is a 
dtMJLSs. Note that in our feedback scheme we assume 
that the controller can measure the whole state x{k) = 
[xp{kyxN{kyy of (I), where xp{k) G is the state of 
the plant and x^ik) G are state variables modeling 

the flow of packets via all routing paths. We assume that 
the controller can measure the state xp{k) of the plant 
via sensors and defer to future work the output-feedback 
case. Also, the controller is aware of the current and past 
actuation signals u{k) that have been sent to the actuator, 
as well as of the current and past signals a{k) (which is 
either constant for any k or choosen by the controller): as 
a consequence the controller has direct access to the state 
of xp[{k)^ which models the actuation commands that are 
expected to arrive at the actuator, but is not aware of their 
arrival to the actuator since (7{k) is not measurable. 


Proof: The proof is constructive and shows how to compute 
K'^{k) for any k = 0, ...,A — I.We start from the 
classical Belmann optimization formulation (Lancaster 
and Rodman (1995), Bertsekas (2005)): 

f J{x{k), u{k)) = 

I min E{x'{k)Mx{k) u {k)Ru{k) J {x{k1), u{k 

J a(k),u(k) 


yj{x{N),u{N)) = J{x{N)) = x' {N)E{Q\^n}x{N) = x' (N)P{N)x{N) 

(3) 

where P{N) = Q is a symmetric matrix and J{x{k)^u{k)) 
is the cost-to-go function at time k. Let us write the cost- 
to-go function at step N — 1: 


J{x{N -l),u{N -1)) = 

min e\ x'(N - l)Mx(N - 1) + u'(N - l)Ru(N - 1)+ 
c{N-l) I 


■} 


u(N-l) 

+ x'(N)P(N)x(N)\i<N.i } = 
min e/ x'(N - l)Mx(N -l) + u'(N - l)Ru(N - 1)+ 

n(V-l) L 

{x'{N - + u'{N - 1)S'- 1) + 


1 )) 


'^N-1 


By linearity of the expected value we can write: 


J{x{N - 1), u{N - 1)) = 

min |£;{cc'(V - l)Mcc(V - 1)|K7 V-i} + 

u(Ar-l) ^ 

E{u'(N - l)Ru{N - l)|Kjv-i} + 

E{{x'(N - - l)\^N-l} + 

E{{x'{N - + 

E{{u'(N - l)P;^_^P(iV)A^(^_l)Cc(iV - l)\^N-l} + 

E{u'(N - - l)|K^r_i}} (4) 







Let us consider each addend of the rightside of (4). Since 
M and R are constant matrices, x{N — 1) is Htv-i- 
measurable, and u{N — 1) is not a random variable since 
it is the input that we choose to apply to the system at 
time N — the first two addends can be written as 

E{x'{N - l)Mx(N - 1) + u'(N - l)Ru{N - = 

x'(N - l)Mx(N - 1) + u'{N - l)Ru(N - 1). 

Moreover, x'{N — 1)Mx{N — 1) does not depend on u{N — 
1) and can be moved out of the min operator. The third 
addend can be written as 

E{{x'{N - 1 )A'^^„_,^P{N)A,^n-i)x{N - l)|Hiv-i} = 
x'{N - - 1) = 

x'{N -1){N -1), 

where ^{N — 1) = XI Since x{N — 1) is 

i=l 

Hat- 1 -measurable, P{N) is symmetric and a{k) are i.i.d., 
the sum of the fourth and fifth addends can be written as 

2E{{x^{N - - l)|Hiv-i} = 

2x'iN - l)E{A'^^j,_,^P{N)\HM-i}Ba,_MN - 1) = 

2x\N - l)E{A'^^^_,^}P{N)Ba^_MN - 1) = 

2x'{N - l)A'P{N)Ba^_MN - 1), 

where A = L^{^a(Ar-i)} = Yli=iAi7Ti. The last addend 
can be written as: 

E{u'{N - l))B',^_^P{N)Ba^_MN - = 

u'{N - l))B'^^_^P{N)Ba^_MN - 1). 

We can now rewrite (4) as follows: 

J(x{N - 1), u{N - 1)) = x'{N - 1) [M + - 1)] x(N - 1) + 

min \ 2 x'{N - l)A'P(N)Ba^_-,u(N - 1 ) + 
u{N-l) 

u'{N - - 1) + u'(N - l)Ru{N - 1)} (5) 

We can compute the minimun of the above function with 
respect to u{N — 1) by equaling the derivative with respect 
to u{N — 1) to 0: 

2x'iN -l)A'PiN)Ba^_^ +2u'iN-l) [b'^^_^P iN)Ba + i?] =0, 

( 6 ) 

which gets to 

u(N -1) = - [R + B'^^_^P(N)Ba^_^]~^ B'^^_^P(N)Ax(N -1). 

We have thus obtained a linear feedback of the state given 
by 

u{N - 1) = KAN - l)x{N - 1) (7) 

By replacing the expression of u{N — l) in the cost function 
(5) it is possible to obtain 

J(x{N-l)) = 

x'(N - 1) [m + $(V - 1) + A'P(N)Baj^_^ [r + BA_^P(N)Ba^_A~^ ■ 
“ b = - 1)P(V - l)x{N - 1). 

By iterating this procedure for a generic k it is possible to 
obtain the expression for K'^{k) 

K*{k) = -[R + B',^P(k + \)Ba,] + 1)A. (8) 

This concludes the proof. □ 

We apply the above theorem to the following System 
(1) characterised by a 4-dimensional unstable randomly 
generated plant 
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and by a wireless network carachterized by two paths: pi 
with packet loss probability pi = 0.25 and delay di = 1 
and p 2 with packet loss probability P 2 = 0 and delay 
^2 = 5. We applied Theorem 1 to three simple routing 
strategies, i.e. using for all time instants only path pi, only 
path p 2 or both paths simultaneously. We computed our 
solution for a time horizon N = 300 and defined the weight 
matrices M, i?, Q and initial condition x(0) as follows: 


If routing via both pi, P2 = 
M = Q = 


IA 0 
0 0 


if routing via p\ : 


M = Q = 


I 4 0 
0 0 


G 


cc(0) = (1, 1, 1, 1, 0, . . . , 0) G I 


if routing via p 2 : 


M = Q = 


I 4 0 
0 0 


G 


R=l, cc(0) = (1, 1, 1, 1, 0, . . . , 0) G K®. 


For each routing strategy we performed 5K MC simula¬ 
tions of the state trajectories. Figure 2 shows the trajec¬ 
tories of the first component of the extended state vector 
when only path pi is used. The system can be stabilized, 
but the variance of the trajectories is high. In many other 
simulations the whole system can’t be stabilized only using 
path pi, many state trajectories are unstable because of 
the very high packet loss probability. This routing policy is 
clearly a bad choice. Figure 3 shows the trajectories when 



Fig. 2 . State trajectories routing only via path pi (blue) 
and their average (red). 

only path p 2 is used (Red) and when both paths pi and p 2 
are used (Blue and green). Routing data only to path p 2 
clearly generates always the same trajectory since P 2 = 0 . 
The system trajectory is stable but the associated cost 
is quite high because of the large delay, as evidenced by 
the overshoot and the settling time performances. Figure 
3 evidences that routing data via both paths pi and p 2 
the control performance strongly improves: in particu¬ 
lar, the trajectory of the system computed by averaging 
over all MC simulations is characterised by much smaller 
overshoot and faster settling time. The single trajectories 
generated routing data via both paths pi and p 2 clearly 
have some variance due to the high packet loss probability 
Pi: however, in the bK MC simulations, the performance of 
any of the single trajectories is much better than the case 
when only path p 2 is used. Table 4 shows the tremendous 
improvement of the controller performance obtained by 
exploiting both paths and co-designing controller gains 






























Fig. 3. State trajectory routing only via path p 2 (red 
dashed); state trajectories routing via both paths pi 
and p 2 (blue) and their average (green). 

and static routing redundancy parameters. Finally, the 



Averaged cost 

Via path pi 

- 900 

Via path p 2 

- 250 

Via paths pi , p 2 

- 100 


Table 1 . Cost averaged over 5K MC sims. 


VLi{k) can be defined by a finite set of quadratic inequalities 
in the form x'Yx ^ 0 , {<, <, >, >} with F a n x n 

symmetric matrix of reals. Moreover, at each step k such 
solution is optimal for a subset L{k) C { 0 ,... ^uj{k)}. 

Proof: The proof is constructive and shows how to compute 
each Qi{k). We start from the classical Belmann optimiza¬ 
tion formulation: 

J{x{k), u(k), a{k)) = 

min E{x' (k)Mx(k) + u' (k)Ru(k) + J{x{k + 1), u{k + 1), a{k + 
a{k),u(k) 

J{x(N),u{N),a(N)) = J(x{N)) = x' (N) E{Q\H n}x(N) = x' (N)P{N)x{N) 

(9) 

where P{N) = Q is a symmetric matrix and J{x{k)^u{k)^ a{k)) 
is the cost-to-go function at time k. We first provide the 
optimal solution at step — 1. Then we provide the 
optimal solution at step N — 2 given the optimal solution 
at step N — 1. The optimal solution at any other step 
k = 1,...,A^ — 3 can be obtained iterating the same 
reasoning of step N — 2. 

Step N — 1: Let us write the cost-to-go function at step 
N -1: 


averaged actuation signals V 2 {k) = U 2 {k — 5) routing via 
path p 2 and vi^ 2 {k) = ui{k — l)-\-U 2 {k — b) routing via both 
paths are shown in Figure 4: note that the actuation cost 
(i.e. energy) associated to the case when we use only path 
p 2 is much larger w.r.t. to the case when we use both paths 
pi and p 2 . Using our approach it is possible to compute. 



Fig. 4. Actuation signal V 2 {k) = U 2 {k — b) (red) routing via 
path p 2 ] averaged actuation signal vi^ 2 {k) = ui(k — 
1 ) Pu 2 {k — 5) (blue) routing via both paths. 

for a finite set of predefined routing policies, the associated 
expected quadratic cost and choose the cheapest policy. 
To further improve the performance one can dynamically 
choose, for each packet and according to the plant state 
measurement, the routing path: this is the problem we 
address in the next section. 


J{x{N - l),u{N - 1), a{N - 1)) = 

min e\ x'iN - l)Mx{N - 1) + u (N - l)Ru(N - 1)+ 

a(V-l),n(V-l) I 

+ x'(Ar)P(iV)x(7V)|Hjv-i| = 

min e\ x'(N - l)Mx(N - 1) + u'(N - l)Ru(N - 1)+ 

(x(iV-l),u(Ar-l) I, ' 

{x'{N - + u'(N - l)Bl(jv-i))^’(^)(V(iV-i)a:(iV - 1)+ 


- 1 )) 


^N- 


} 


By linearity of the expected value we can write: 

J{x{N - 1), u{N - 1), a{N - 1)) = 

min |£;{cc'(V - l)Mcc(iV - l)|Kjv-i} + 

E{u'{N - l)Ru{N - l)|^^Ar_i} + 

E{{x'{N - 1)T - l)|K^^-l} + 

E{{x'{N - - l)|^Ar-i} + 

E{{u'{N - + 

E{u'{N - l))<(^_,)P(iV)P,(^_i)n(iV - (10) 

Let us consider each addend of the rightside of (10). 
Since M and R are constant matrices, x{N — 1) is Hat-i- 
measurable, and u{N — 1) is not a random variable since 
it is the input that we choose to apply to the system at 
time N — the first two addends can be written as 

E{x'(N - l)Mx(N - 1) + u'{N - l)Ru(N - = 

x'{N - l)Mx{N - 1) + u'{N - l)Ru(N - 1). 


5. CO-DESIGN OF CONTROLLER AND DYNAMIC 
ROUTING REDUNDANGY 


Moreover, x'{N — l)Mx{N — 1) does not depend on a{N — 
l),u{N — 1) and can be moved out of the min operator. 
The third addend can be written as 


In this section we provide a suboptimal solution of Prob¬ 
lem 1 that is optimal for a certain set of initial conditions, 
which we define in closed form. 

Theorem 2. Given System ( 2 ), a solution of Problem 1 is 
given by the sequence, Mk = 0 , ...,Y — 1 , of a*(x(/c)), 

K*{x{k)) defined by a finite partition Tt{k) = {^i{k)}^^^ 
of each assocated with a pair Oi^Ki such that 

a*(x(/c)) = ai,K'"{x{k)) = Ki x{k) G Ttiik). Any 


E{{x\N - 1)A;(^_i)P(Y)A,(^_i)x(Y - mN-i} = 
x\N - 1)P{A;(^_i)P(7V)A,(^_i)|H^_i}x(7V -1) = 
P(Ar- 1)4>(Y- l)(Ar- 1), 

where ^{N — 1 ) = A'-P{N)Ai7ri. Since x{N — 1 ) is 

KTV- 1 -measurable, P{N) is symmetric and a{k) are i.i.d., 
the sum of the fourth and fifth addends can be written as 

























2E{{x'{N - - l)|H,v-i} = 

2x'{N - l)E{A',^j,_^.^P{N)\i<M-i}B,iN-i)u{N - 1) = 
2x'{N - l)i;{<(^_i)}P(7V)B,(jv-i)w(iV - 1) = 

2x'{N - l)A'P{N)Ba(N-i)u{N - 1), 

where A = ^{^cr(Ar-i)} = The last addend 

can be written as: 

E{u\N - - l)|H^_i} = 

u\N - l))P;(^_i)P(Ar)5,(^_i)^(7V - 1). 


J{x{N - 2), u{N - 2), a{N - 2)) = 

min e\x'{N - 2)Mx{N - 2) + u'{N - 2)Ru{N - 2) + 

a(N-2),u(N-2) ^ 

x'(N - 1)P(N - l)x(N - 1)|Kjv-2}, 

By linearity of the expected value we can write: 

J{x(N - 2), u(N - 2), a{N - 2)) = 

min \ E{x'(N - 2)Mx{N - 2)\nN_2} + 

a(N-2),u(N-2) ^ 

E{u'{N - 2)Ru{N - 2)|Kjv-2} + 

E{(x'{N - 2)A;(^_2)P(V - 1)A^(^_2)Cc(V - 2)|K^,_2} + 
E{{x'{N - 2)A;(^_2)P(V - l)n,(^_ 2 )n(V - 2)|K^r-2} + 


We can now rewrite (10) as follows: 

J(x{N - 1), u{N - 1), a{N - 1)) = x'(N - 1) [M + - 1)] x{N - 1) + 


E{{u'{N - 2 )B'^^j^_^^P{N - 1)A^(^_2)Cc(V - 2)\^n-2} + 

E{v.'(N - 2))B;(„_2)P(JV - I)i3<.(;v-2)«(JV - 2)|N„_2}} (14) 


min I 2x'{N - l)A'P{N)B^rj^_^)u{N - 1) + 

a{N-l),u(N-l) ^ 

u'{N - - 1) + u'{N - l)Ru{N - 1)} (11) 

For any given a{N — 1) G A we can compute the minimun 
of the above function with respect to 14(A^ — 1 ) by equaling 
the derivative with respect to u{N — 1) to 0: 

2x'{N - 1)A'P(V)P„(^_1) + 2u'{N - 1) [p;(^_,)P(V)P„(^_l) + P] =0, 

(12) 

which gets to 

^a(N-l)(N - 1) = - [P + P;(^_,)P(V)P„(^_1)] B'^^j^_^^P(N)Ax(N - 

We have thus obtained a linear feedback of the state given 
by 

UaiN-l){N - 1) = KaiN-l){N - l)x{N - 1) (13) 

By replacing the expression of Ua(N-i){^ ~ 1) fbe cost 
function ( 11 ) it is possible to obtain 

J(x(N - 1), a(N - 1)) = 

x'iN - 1) [m + $(V - 1) + A'P(V)P^(^_ 1 ) (P + <(^_,)P(V)P,(^_ 1 )) 

- 1) = ^'(N - l)Pa(N-l)iN - l)x{N - 1). 


Let us consider each addend of the rightside of (14). 
Since M and R are constant matrices, x{N — 2 ) is '^n- 2 - 
measurable, and u{N — 2) is not a random variable since 
it is the input that we choose to apply to the system at 
time N — 2, the first two addends can be written as 

E{x'(N - 2)Mx{N - 2) + u'{N - 2)Ru{N - 2)|Kjv-2} = 
x'{N - 2)Mx{N - 2) + u'{N - 2)Ru{N - 2). 

Moreover, x'{N — 2)Mx{N — 2) does not depend on a{N — 
2),u{N — 2) and can be moved out of the min operator. 
Note that P{N — 1) depends, according to the optimal 
control policy Q{N — 1), on the random variable x{N — 
^^1), which in turn depends on a{N — 2),x{N — 2),a(N — 
2),u{N — 2). As a consequence, differently from step N — 1, 
computing the expected values in Equation (14) is non¬ 
trivial. The main idea of this proof is to overcome this 
difficulty, instead of exploiting the conservative approxi¬ 
mation used in Vargas et al. (2010), by considering that 
such expected values can assume a finite number of values. 
In particular, given a discrete control input a{N — 2) = a, 
a linear feedback K{N — 2) = K and a state x{N — 2)=x 
■ the probability that P{N — 1) = Pi{N — 1) is equal to 

E 


Let us now define a partition of given by the collection 
of disjoint sets Q{N — 1) = {Cli{N — l)}f=i where each 
fli{N — 1) is defined by 

ni(N - 1) = {x(N - 1) e : i = arg min J(cc(V - 1), a(V - 1))}. 

a(N-l) 

fli{N — 1) is the set of all states x{N — 1) such that 
the corresponding optimal discrete control is a*(x(/c)) = 
K'^{x{k)) = Ki{N — 1 ) and can be defined by the 
following set of inequalities: 

(x'{Pi{N - 1) - Pi{N - l))x < 0 


x'(P,(V-l)-P,_i(V-l))x<0 

x'(P,(V-l)-P,+i(V-l))x<0 


[x'{Pi{N - 1) - Pp{N - l))x < 0 

where the matrices are all symmetric. Note that Vtik) = 
{fli{N — l)}f=i is by definition a partition of W^. Each 
(}i{N — 1) is associated with the discrete control i, the 
continuous state-feedback control Ki{N — 1) and the cost 
Pi{N — 1). Note that, at this step, each Qi{N — 1) provides 
the optimal solution, i.e. l{N — 1) = {1,... ,p}. 


c7:(Aa+BaK)xeniiN-l) 

Let fj, = {lXa}ae{i,...,q} bo a vector of q natural numbers 
ficr G {1,..., uj{N — 1)} representing all possible combina¬ 
tions of sets fti{N — 1) towards which x{N — 2) can be 
driven by the occurrence of all a G E. It is easy to see 
that, given any /L 6 , the expected values in Equation (14) 
can be computed. In particular, the third addend can be 
written as 

E{ix'(N - 2)A;(^_2)P(iV - 1 )A,(^_ 2 )^(A^ - 2)\^n-2} = 
x'iN - 2)£;{A;(^_2)P(iV - l)A^^N_ 2 )\^N- 2 }xiN - 2) = 

x'{N - 2) (N - l)A<,7r, j x(N - 2) (15) 

Since x{N — 2) is Hat- 2 -measurable, Pi(V — 1 ), i = 1 ,... ,p 
are all symmetric and (j{k) are i.i.d., the sum of the fourth 
and fifth addends can be written as 

2E{{x'{N - 2)A;(^_2)P(V - l)n^(^_ 2 )n(V - 2)\i<N_2} = 

= 2x'{N - 2)£;{A;(^_2)P(V - l)|KiV-2}Pa(iV-2)^(^ - 2) = 

= 2x'{N - 2) KPpa Pa(JV-2)^(^ - 2). (16) 

The last addend can be written as: 

E{u\N - 2))n;(^_2)P(V - l)Pa(A,_2)n(V - 2)\^n- 2} = 
u'(N - 2))n;(^_2)P{P(V - l)\^N- 2 }Ba(N- 2 )^(N - 2) = 


Step N — 2: Given the optimal solution Q{N — 1) of step 
V — 1 we provide the optimal solution at step N — 2. Let 
us write the cost-to-go function at step N — 2: 


n'(V- 2 ))<(^_ 2 ) 




,(N - l)7r<, B„(N_2)u(iV - 2). 


(17) 





We can now, for all states x{N — 2) that are driven by each 
cr G S to the set — 1), rewrite (14) as follows: 


J{x{N - 2), u{N - 2), a{N - 2), fx) = 


min I x (N — 2) 
a(N-2) ^ 


x(N - 2) + 


in^ \^2x\N -2) j (N -1)7T^ j n„(^_2)n(V - 2) + 


mm 

u(N-2) 


n'(V-2))n;(^_2) n(V - 2) + 

n'(V - 2)Pu(V - 2) j j (18) 


As a consequence the optimal linear feedback strategy can 
be computed as in step N — 1 by: 


Ka 


,P' — 


R + b'^ 



(N - l)7Ta 



E 


a;p.(v- 1)71 


A. 


(19) 

The set of states x{N — 2) such that is indeed optimal 

is given by 


r(a, m) = {ic G : VcT G {1, . . . , g}, (A^ + BaKa,t^) G (N - 1)}. 

Given the definition of Q{N — 1) of step N — 1, then r(a, /x) 
can be defined by 

( x'[{Ai _ 1) _ ppAT - l))(Ai PBaKa,^J,^^x < 0 


x'[{Ai + (N -1)- Pp{N - l))(Ai + BaKa,^,^^X < 0 

< : 

x'[{Aq + BaKa,^J,^^Y{Pp.q{N - 1) - Pi(V - l))(Ag + BaKa,tj.r^x < 0 
^x'[{Aq + BaKa,^^^^]\P^,qiN - 1) - Pp{N - l))(Ag + BaKa,^^)~^X < 0 

Note that, for any given a G A, T{a,ix) is a subset of 
W^: its complement r^(a,/L 6 ) can be easily defined by 
replacing in ( 2 ) < and < respectively with > and >, and 
represents a set where the optimal linear feedback strategy 
cannot be computed as in (19). Also, for any given a e A 
and any fi, fi'^ the intersection r(a,/L 6 ) fl r{a,/x') is not 
necessarily empty. Define now the set of disjoint 

sets given by all possible combinations of intersections of 
sets r(a, /L^), a G A, /L 6 , and such that for all d G A at least 
a set r(d, fi) belongs to the intersection. As a consequence 
each can be defined as a finite intersection of sets 
r(a, /L 6 ) and r^(a', /l^'), and can be therefore characterised 
by a finite set of quadratic inequalities. Consider now a 
generic set 

fii = r(ai, ^i.i)nr(a2, M2)n- ■ ■nr(a^, fu,z)r\r^ Mz+i)gi- ■ ■nr^(a^, p-y). 

For any x{N — 2) G we have the choice to apply a finite 
number of optimal control pairs {{aj, Kaj,^j)}je{i,...,z}i 
each associated to the optimal cost 

J(x(N - 2), aj ,^Xj)= x\N - 2)Pa. {N - 2)x{N - 2). 

To choose the optimal control feedback within we 
partition it, as in step N — with a finite collection of 
disjoint sets defined by 

= {x(N — 2) G : a,/x = arg min J{x(N — l),a, /x)}. 

a,/x 

is the set of all states x{N — 2) such that the 
corresponding optimal control is a*(x(/c)) = a, K''{x{k)) = 
As a consequence can be defined as follows: 


- 1) - - 1))X < 0 

o ^ I - 1) - - l))a: < 0 

* ' A x'{Pa„^^{N - 1) - - 1))X < 0 ’ 

(iV - 1) - - 1))X < 0 

(20) 

where the matrices are all symmetric. Note that }|=i 

is by definition a partition of Qi. To each is associ¬ 
ated the control a^, and the cost Applying 

the same partition to each Cli provides the optimal solution 
for any state in the set = |J E The set of 

states \ fl such that the optimal solution cannot 

be computed using the method above can be also defined 
as a finite union of sets of quadratic inequalities. To com¬ 
pute a suboptimal solution for states x{N — 2) G fl^{N — 2) 
we can partition it in a finite number of sets fla,^ ^s in (20) 
according to the set of all pairs Of course this 

will not be the optimal solution, since by definition there 
exists at least a discrete control a such that is not 

the corresponding optimal linear feedback. We can define 
fl{N — 2) by the union of all sets that partition fl and fl^^ 
as defined above. fl{N — 2), together with the associated 
discrete controls, linear feedback gains and costs, will be 
provided as input to step N — 3. Also, the set l{N — 2) 
can be easily characterised since only the solutions for the 
sets partitioning fl are optimal. The solution at any other 
step k = , N — 3 given the solution at step k 1 can 

be obtained iterating the same reasoning of step N — 2. 
The only difference is that a set fli^a,fi is associated to 
an optimal solution only if fl^^{k + 1 ) is associated to an 
optimal solution for all a G E. Once arrived to the initial 
step A" = 0, we can define the sets of initial conditions 
such that the strategy is optimal. This concludes the proof. 

□ 


6 . CONCLUSION 

This paper proposes a novel paradigm of networked control 
where the exploitation of redundancy in routing actuation 
data tremendously improves the control performance, by 
exploiting in an optimal way the advantages of different 
paths characterised by propagation characteristics that 
are at odds one another. We show that the co-design 
of controller and routing redundancy, given a routing 
that is defined a priori, strongly improves the control 
performance. We also provide a methodology to co-design 
controller and dynamic routing redundancy: note that, 
according to our solution, routing depends on the plant 
state, and it is even possible to decide not to route any 
actuation data for some time instants, which generates an 
event-triggered control strategy to send actuation data not 
at all time steps, but only when necessary according to the 
current plant state. In future work we plan to implement 
a tool that applies our algorithms and to extend our 
methodology to more general classes of stochastic systems. 
Also, we will apply the methodologies in this paper to 
address the dynamic scheduling and routing co-design in 
communication protocols for wireless control systems such 




as WirelessHART and ISAIOO (see R. Alur et al. (2011) 

and D’Innocenzo et al. (2013) for details). 
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